Date: Tue, 10 Dec 1996 15:17:36 GMT
Server: NCSA/1.4.2
Content-type: text/html
Last-modified: Mon, 26 Sep 1994 21:34:38 GMT
Content-length: 2151

<html>
<head>
<title>
Testbeds and Evaluation
</title>
</head>

<body>
<h3>Planning testbeds and empirical evaluation of agents </h3>

<ul>
     <li> <a name="truckworld"> The Truckworld testbed </a> is a challenging 
       software environment to exercise your planning agent.  It 
       allows you to model
         <dir>
          <li> multiple agents that can communicate and cooperate
          <li> a rich set of objects that the agent can manipulate 
          <li> exogenous events
          <li> actions that fail randomly and in a state-dependent way
          <li> complex sensing and communication devices 
          <li> easy customization to your problem domain.
         </dir>
      <li> The  <A HREF="ftp://ftp.cs.washington.edu/pub/ai/tr93-06-09.ps.Z">
                  Truckworld Beginner's guide </a> 
                 provides an overview of the 
                 system's basic functionality, and tells you how to set up
                 the system and run a simulation.
          <li> The Truckworld code is available via anonymous FTP or 
                by clicking 
                  <A HREF="ftp://ftp.cs.washington.edu/pub/ai/truckworld.tar.Z">
                    here</a>,  (but send mail 
                to <tt> truckworld-request@cs.washington.edu </tt> 
                if you use the code, so we can send you updates and 
                bug fixes).
         <ul>
     <li> The paper 
          <A HREF="ftp://ftp.cs.washington.edu/tr/1993/06/UW-CSE-93-06-05.PS.Z">
            <i> Benchmarks, Testbeds, Controlled Experimentation and the Design of 
                  Agent Architectures</i> </a>
          is a broader overview of the issues involved with building good testbeds 
          and evaluating an agent's performance.  This paper appeared in 
         <it> AI Magazine </it> Winter 1993, and is joint 
         work with  
             <a href="http://www.cs.pitt.edu/~pollack/">  Martha Pollack </a>
                and 
             <a href="http://eksl-www.cs.umass.edu/~cohen/home.html"> Paul Cohen</a>. 
         </ul>
     </ul>


<address> 
hanks@cs.washington.edu
</address> 

</body>
</html>
